THE GEORGE WASHINGTON UNIVERSITY 



CENTER for EQUITY and 
EXCELLENCE in EDUCATION 


Examination of Peer Review and Title I Monitoring 
Feedback Regarding the Inclusion and Accommodation of 
English Language Learners in State Content Assessments 


Lynn Shafer Willner, Charlene Rivera, and Barbara D. Acosta 


The George Washington University Center for Equity and Excellence in Education 

1555 Wilson Boulevard, Suite 515, Arlington, VA 22209 
Tel: 703.528.3588 Fax: 703.528.5973 
www.ceee.gwu.edu 


The George Washington University 
Center for Equity and Excellence in Education 


The mission of The George Washington University Center for Equity and Excellence in Education (GW- 
CEEE) is to advance education reform so all students achieve to high standards. GW-CEEE conducts 
policy and applied research, designs and implements program evaluations, and provides professional 
development and technical assistance. GW-CEEE’ s clients include state education agencies, school 
districts, schools, foundations, and federal agencies. 

Eor over a decade, GW-CEEE has conducted research on the inclusion and accommodation of English 
language learners (ELLs) in high stakes testing, including periodic reviews of state assessment policies 
for ELLs. GW-CEEE is currently conducting an applied research project with support from the Bill & 
Melinda Gates Eoundation to study the academic language demands of ELLs in middle and high schools. 
GW-CEEE also provides technical assistance and conducts professional development for clients in states, 
districts, and schools. With funding from the U.S. Department of Education, GW-CEEE operates the 
Mid-Atlantic Comprehensive Center which provides technical assistance to state education agencies in 
the Mid-Atlantic region. GW-CEEE also operates the Mid-Atlantic Equity Assistance Center which 
provides trainings and technical assistance to districts and schools in Delaware, the District of Columbia, 
Maryland, Pennsylvania, Virginia, and West Virginia. 


This report is in the public domain. While permission to reprint this publication is not necessary, it should 
be cited as: 

Shafer Willner, L., Rivera, C., & Acosta, B. (2010). Examination of Peer Review and Title I Monitoring 
Feedback Regarding the Inclusion and Accommodation of English Language Learners in State 
Content Assessments. Arlington, VA: The George Washington University Center for Equity and 
Excellence in Education. 

This report is available on the GW-CEEE website: www.ceee.gwu.edu. 

This report was supported by funds from the U.S. Department of Education. The views expressed herein 
do not necessarily represent the official positions or policies of the Department of Education. No official 
endorsement by the U.S Department of Education is intended or should be inferred. The contents are the 
sole responsibility of the authors. 

Copyright © 2010 by The George Washington University Center for Equity and Excellence in Education 


GW-CEEE 

1555 Wilson Boulevard, Suite 515 Arlington, VA 22209 
www.ceee.gwu.edu 



Acknowledgements 


This publication was supported by funding from the U.S. Department of Education (ED). We are grateful 
for the collaboration among ED offices that believed it was important to fund work in this area. In 
particular we would like to thank Jenelle Leonard, Director, School Support and Technology Programs; 
Fran Walter, Group Leader, Technical Assistance Programs; and Richard Smith, Acting Director for the 
Office of English Language Acquisition (OELA) for supporting the project. We are also grateful to SEA 
assessment directors and Title III directors in 49 states who participated in the monitoring survey, many 
of whom generously shared examples of monitoring tools and resources. 

Special recognition goes to GW-CEEE staff who provided important support throughout the project. 
Roshaun Tyson provided report design and layout expertise; Jennifer Lynch and Gamaliel Gabriento 
provided administrative support; and Charlotte Blane provided editorial expertise. 


The George Washington University Center for Equity and Excellence in Education I www.ceee.gwu.edu 



Table of Contents 


Executive Summary v 

Introduction 1 

Related Research 1 

Method 2 

State Standards and Assessment Peer Review Requirements Addressing ELLs 2 

Title I Monitoring Requirements Addressing ELL Inclusion and Accommodation 4 

Data Collection and Analysis 7 

Limitations of this Study 7 

Findings 8 

Issues Pertaining to the Inclusion of ELLs 8 

Issues Pertaining to the Accommodation of ELLs 10 

Discussion 12 

Recommendations 14 

Recommendations for SEAs 14 

Recommendations for ED 15 

Conclusion 15 

References 16 


The George Washington University Center for Equity and Excellence in Education I www.ceee.gwu.edu 



List of Tables 


Table 1. Peer Review Components and Critical Elements Pertaining to ELLs 3-4 

Table 2. Summary of Title 1 Monitoring Indicators and Guiding Questions Pertaining to ELLs 5-6 

Table 3. Title I Monitoring Visits during tbe Tbree-Year Monitoring Cycle, 2006-2009 6-7 

Table 4. Number of SEAs Receiving Eeedback on tbe Inclusion and Accommodation of ELLs in Peer 

Review Decision Letters and Title I Monitoring Reports 8 


The George Washington University Center for Equity and Excellence In Education I www.ceee.gwu.edu 


IV 



Examination of Peer Review and Title I Monitoring Feedback Regarding the Inclusion and 
Accommodation of ELLs in State Content Assessments 

Executive Summary 

The purpose of this study was to investigate the extent to which English language learners (ELLs) are 
being offered meaningful opportunities to participate in state content assessments. Unfortunately the 
current collection and analysis of data about the inclusion of ELLs in accommodated and 
unaccommodated state assessments are, at best, variable and incomplete. As there is no requirement to do 
so, state education agencies (SEAs) do not usually track the extent to which ELLs are or are not assessed 
with accommodations. 

Two types of review, standards and assessment peer review and the Title I monitoring review, conducted 
by the Department of Education (ED), are intended to support SEAs in meeting the requirement to validly 
assess all students including ELLs and students with disabilities as required by the No Child Left Behind 
Act of 2001 (NCLB), the current reauthorization of the Elementary and Secondary Education Act (ESEA) 
of 1965. The purpose of peer review is to provide individualized feedback to each SEA on the adequacy 
of its standards and assessment system in reading/language arts, mathematics, and science. In contrast, the 
function of the Title I monitoring review is to examine the adequacy of implementation of Title I 
requirements by an SEA. In the area of assessment, peer reviews focus primarily on assessment design 
and valid use, while Title I monitoring reviews include a component that centers on the implementation of 
assessments. Thus, the amount and types of feedback differ for the two data sources, reflecting the 
differing purposes of the two types of review. 

The study was conducted to examine the extent to which standards and assessment peer review decision 
letters and Title I monitoring reports provide feedback to SEAs about their policies and practices for 
including and accommodating ELLs in state assessments. In the investigation the research team reviewed 
two sets of documents taken from ED’s Web site (www.ed.gov) . These included 273 peer review decision 
letters to SEAs issued to 52 SEAs (DC and PR included) from June 17, 2005 to January 15, 2009, and the 
Title I monitoring reports issued to 52 SEAs during the most recently-completed three-year monitoring 
cycle of 2006-2007, 2007-2008, and 2008-2009. 

Eeedback from the peer review letters to SEAs and Title I monitoring visit reports were analyzed to 
address the research questions guiding this investigation: 

1. To what extent do SEAs receive feedback in peer review decision letters and/or Title I monitoring 
reports that address the inclusion and accommodation of ELLs? 

2. What issues are identified in peer review decision letters and Title I monitoring reports to SEAs 
related to the inclusion and accommodation of ELLs? 

The data examined for this study are of course influenced by the limitations of the individual procedures 
established for each review, the quality of the data provided by SEAs for reviewers, and the quality of the 
feedback provided by reviewers. 

Two significant findings emerged from this study. The primary finding is that the majority of SEAs have 
weaknesses in their policies and practices for including and accommodating ELLs in state assessment 
systems. Secondly, the feedback itself is inconsistent both within and across the peer review and Title I 
monitoring processes. Overall, inclusion accommodation issues addressed in peer review letters and Title 
I monitoring reports suggest the need for SEAs to reassess their policies and to find methods to uniformly 
implement them across a state. The wide range of ELL inclusion and accommodation feedback and the 
lack of similarity and detail in feedback given to SEAs however, points to a lack of consistency and 
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coherence in the evidence provided to ED hy SEAs as part of peer review and Title I monitoring review. 
The lack of consistency and coherence in the data provided hy SEAs for the two reviews suggests a need 
for a concerted effort to clarify for states what is considered acceptable evidence. 

The findings also have important implications for the two review processes. The inconsistencies within 
and across decision letters and Title I monitoring reports point to the need for greater alignment between 
peer review and Title I monitoring. Also, there is a need to select reviewers familiar with the assessment 
of ELLs and to train review teams so all members have a common understanding of issues related to the 
inclusion and accommodation of ELLs, even if it means increasing the size of the team. 

In conclusion, because SEAs’ standards and assessment systems are at different stages of development 
and different levels of complexity, this analysis points to a number of suggestions for improving the 
coherence, quality, and effectiveness of both SEA and ED practice. Efforts within ED could lead to 
greater understanding, and therefore improved compliance of SEAs, more coherent SEA plans, and 
ultimately more meaningful inclusion and accommodation of ELLs in state assessment. 
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Examination of Peer Review and Title I Monitoring Feedback Regarding the Inclusion 
and Accommodation of ELLs in State Content Assessments 

Introduction 

The No Child Left Behind Act of 2001 (NCLB), the current reauthorization of the Elementary 
and Secondary Education Act (ESEA) of 1965, requires the valid assessment of all students 
including English language learners (ELLs) and students with disabilities (SDs). In order to 
determine the extent to which ELLs are being offered meaningful opportunities to participate in 
state content assessments, a thorough examination of state education agencies’ (SEAs) policies 
and practices for including and accommodating ELLs in state assessments is needed. 

SEAs report inclusion of ELLs in state assessments in Consolidated State Performance Reports. 
However, as pointed out by the Government Accountability Office (GAO), these reports must be 
interpreted with caution because of incomplete data and inconsistencies in data collection (United 
States Government Accountability Office, 2006, p. 2). SEAs do not usually track the extent to 
which ELLs are or are not assessed with accommodations because there is no Consolidated State 
Performance Report requirement to do so. 

Currently the Department of Education (ED) conducts two reviews intended to support SEAs in 
meeting the requirement to validly assess all students: standards and assessment peer review and 
Title I monitoring review. This investigation identifies the extent to which these two review 
processes provide consistent feedback to SEAs about their policies and practices for including 
and accommodating ELLs in state assessments. 


Related Research 

Two descriptive reviews of peer review decision letters preceded this study. A Regional 
Educational Laboratory Central (REL Central) review examined feedback in peer review decision 
letters related to the assessment of students generally (Palmer & Barley, 2007). A review by the 
National Center for Educational Outcomes (NCEO) examined peer review feedback for issues 
specific to students with disabilities (Christensen, Thurlow, & Wang, 2009; Christensen, Lail, & 
Thurlow, 2007). 

Palmer and Barley, as a result of reviewing decision letters (posted for SEAs through Eebruary 
28, 2007) and interviewing a small group of SEA assessment directors, identified the challenges 
of preparing materials for peer review and created seven detailed checklists. Each chec kl ist 
addresses one of the seven peer review components, which highlight acceptable and unacceptable 
forms of evidence. Christensen and her colleagues’ review of 2006-2007 decision letters resulted 
in “hints and tips” for SEAs in preparing peer review materials addressing the accommodation of 
students with disabilities. Both groups of researchers note the varying levels of detail and 
inconsistency among peer reviewer comments. 

Another piece of related research is a state survey of state monitoring practices conducted by The 
George Washington University Center for Equity and Excellence in Education (GW-CEEE) in 
March 2009. The results from the survey suggested that states are collecting limited data about 
the extent to which ELLs are included and accommodated in assessment, which has implications 
for the quality of data submitted to ED for peer review and Title I monitoring reviews. 
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Method 

Feedback from peer review letters to SEAs and Title I monitoring visit reports were analyzed to 
address the research questions guiding this investigation: 

1. To what extent do SEAs receive feedback in peer review decision letters and/or 
Title I monitoring reports addressing the inclusion and accommodation of ELLs? 

2. What issues are identified in peer review decision letters and Title I monitoring 
reports to SEAs related to the inclusion and accommodation of ELLs? 


State Standards and Assessment Peer Review Requirements Addressing ELLs 

NCLB requires that every SEA administer annual assessments in grades 3-8 and once in high 
school in reading/language arts, mathematics, and science. Prior to the 2005-2006 school year, all 
SEAs were required to provide evidence of the adequacy of their assessment systems for 
reading/language arts and mathematics to a panel of peers. Most SEAs submitted an initial set of 
evidence for peer review of their reading/language arts and mathematics assessments between 
January 2005 and April 2006 and provided additional evidence as requested. SEAs were not 
required to meet the science assessment requirements until the 2007-2008 school year. 

The purpose of peer review is to provide individualized feedback to each SEA on the adequacy of 
its standards and assessment system in reading/language arts, mathematics, and science. A three 
person team is convened to review evidence and to comment on the adequacy of the SEA’s 
standards and assessment system in each content area. Eor each review, SEAs organize evidence 
according to seven components outlined in the guidance, and peers examine the evidence and 
generate comments and recommendations. The peer comments, organized according to the seven 
components outlined in guidance, are incorporated into a letter that is sent to the SEA by the 
Secretary of Education. Based on peer review, as of January 2009, 39 of 52 SEAs had received 
Full Approval or Full Approval with Recommendations for reading/language arts and 
mathematics and/or science assessments (U. S. Department of Education, 2009a). 

Peer reviewers receive general training on the guidance, but do not receive in-depth training on 
the critical elements pertaining to ELLs. Each peer review team evaluates the evidence and 
determines the degree to which an SEA’s final standards and assessment system complies with 
the requirements of NCLB. The peers’ feedback is intended to help SEAs improve their 
assessment systems. Each peer review team generally includes a psychometrician, a special 
educator or an educator knowledgeable about English language learners, and a testing practitioner 
with extensive practical experience in large-scale testing. To provide continuity for each SEA, the 
members generally remain on a team until the SEA achieves passing status. 

Peer review guidance itself is based, in part, on the Standards for Educational and Psychological 
Testing (American Educational Research Association, American Psychological Association, & 
National Council on Measurement in Education, 1999). Each of the seven components of the 
guidance lists critical indicators. As shown in Table 1, two of seven components (4.0 and 6.0) 
developed for peer review target inclusion and accommodation of ELLs. In (4.0) Technical 
Quality, two critical elements relate to the accommodation of ELLs (4.3 and 4.6). In (6.0) 
Inclusion, two critical elements (6.1 and 6.3) address the inclusion of ELLs (and ah students) in 
state assessments and indirectly address monitoring the accommodation of ELLs by requiring the 
SEA to ensure the reliability and validity of assessments used. The highlighted column in Table 1 
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lists the critical elements addressing the inclusion and accommodation of ELLs, the elements of 
focus for this study. 

Table 1 


Peer Review Components and Critical Elements Pertaining to ELLs 


Component 

Description 

Critical Elements Pertaining to ELLs 

1.0 Academic 
content 
standards 

States must develop a set of 
challenging academic content 
standards that define what all public 
school students in the State are 
expected to know and be able to do. 


2.0 Academic 

achievement 

standards 

States must develop a set of 
challenging academic achievement 
standards for every grade and content 
area assessed. 


3.0 Statewide 
assessment 
system 

States must develop a single statewide 
system of high quality assessments. 


4.0 Technical 
quality 

States must employ processes to ensure 
that assessments and use of results are 
appropriate, credible, and technically 
defensible. 

4.3 Has the State ensured that its assessment system is fair 
and accessible to all students, including students with 
disabilities and students with limited English proficiency, 
with respect to each of the following issues: 



(b) Has the State ensured that the assessments provide an 
appropriate variety of linguistic accommodations for students 
with limited English proficiency? and 

(c) Has the State taken steps to ensure fairness in the 
development of the assessments? and 

(d) Does the use of accommodations and/or alternate 
assessments yield meaningful scores? 



4.6 Has the State evaluated its use of accommodations? 

How has the State ensured that appropriate accommodations 
are available to limited English proficient students and that 
these accommodations are used as necessary to yield accurate 
and reliable information about what limited English proficient 
students know and can do? 

5.0 Alignment 

A State’s system of standards and 
assessments must be aligned in order to 
provide useful information for valid 
accountability decisions and 
educational improvement. 

(d) How has the State determined that scores for limited 
English proficiency students that are based on accommodated 
administration circumstances will allow for valid inferences 
about these students’ knowledge and skills and can be 
combined meaningfully with scores from non-accommodated 
administration circumstances? 

6.0 Inclusion 

States must include gU students in a 
State’s system of standards, 
assessments, and accountability. 

6.1 Do the State’s participation data indicate that all students 
in the tested grade levels or grade ranges are included in the 
assessment system (e.g., students with disabilities, students 
with limited English proficiency, economically disadvantaged 
students, race/ethnicity, migrant students, homeless students, 
etc.)? 



6.3 What guidelines does the State have in place for including 
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Table 1 


Peer Review Components and Critical Elements Pertaining to ELLs 


Component 

Description 

Critical Elements Pertaining to ELLs 



all students with limited English proficiency in the tested 
grades in the assessment system? 



(a) Has the State made available assessments, to the extent 
practicable, in the language and form most likely to yield 
accurate and reliable information on what these students 
know and can do? 

(b) Does the State require the participation of every limited 
English proficient student in the assessment system, unless a 
student has attended schools in the US for less than 12 
months, in which case the student may be exempt from one 
administration of the State’s reading/language arts 
assessment? 

(c) Has the State adopted policies requiring limited English 
proficient students to be assessed in reading/language arts in 
English if they have been enrolled in US schools for three 
consecutive years or more? 

7.0 Reporting 

States must provide reports to parents, 
educators, or other stakeholders so that 
they may find answers to questions 
about how well a student or group of 
students is achieving, as well as 
important information on how to 
improve achievement in the future. 



Source: U.S. Department of Education (2009a) Standards and Assessment Peer Review Guidance 


Title I Monitoring Requirements Addressing ELL Inclusion and Accommodation 

In contrast to peer review, the purpose of the Title I monitoring review is to examine the 
adequacy of the implementation of Title I requirements hy an SEA. On a three -year cycle, Title I 
monitoring teams, consisting of Title I staff from the ED Student Achievement and School 
Accountability Programs (SASA) office, conduct site visits to an SEA and selected districts and 
schools. The purpose of the site visit is to examine SEA compliance in three areas: (1) 
Assessment and Accountability; (2) Program Improvement, Parental Involvement, and Options', 
and (3) Fiduciary Responsibilities. The Title I monitoring team assigned issues a report that 
provides feedback in the three areas. The focus for this study was on the examining the feedback 
provided to SEAs in the first area as it relates to the inclusion and accommodation of ELLs (U.S. 
Department of Education, 2009b). 

In Title I monitoring, a team of ED staff conducts site visits to examine SEA compliance with 
Title I grant requirements. As stated on the ED Web site, “A monitoring review is the regular and 
systematic examination of a State's administration and implementation of a Eederal education 
grant, a contract or a cooperative agreement administered by ED.” Monitoring teams use 
performance indicators to assess the extent to which SEAs provide guidance for local educational 
agencies (LEAs) and schools to implement policies and procedures that comply with the 
provisions of Title I. The goal is to gather accurate data about SEA and local needs and to use 
these data to design technical assistance initiatives and national leadership activities. 

The Title I monitoring team focuses on an SEA’s implementation of policies, systems, and 
procedures that ensure LEA and school level compliance with the Title I statute and regulations. 
The extent to which the SEA is in compliance with requirements for including and 
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accommodating ELLs is outlined in the Assessment and Accountability provisions in Title I, Part 
A (U.S. Department of Education, 2009b). 

Eor the 2006-2007 Title I monitoring cycle, Title I guidance focused on six indicators. In 2007- 
2008, the six indicators were expanded to nine. While the contents of indicators 1.2-1. 6 remained 
unchanged, indicator 1.1 was expanded into four indicators. Table 2 displays the component, 
description, and guiding questions (highlighted) pertaining to the inclusion and accommodation 
of ELLs. 

Table 2 

Summary of Title 1 Monitoring Indicators and Guiding Questions Pertaining to ELLs 
Component Description Guiding Questions Pertaining to ELLs 

1.1* The SEA has approved a system of What guidelines does the State have in place for including all 

academic content standards, and students with limited English proficiency (LEP) in the tested 

academic achievement standards and grades in the assessment system? 
assessments (including alternate Documentation: 

assessments) for all required subjects »The State’s guidelines, as communicated to LEAs, call for the 
and grades, or has an approved timeline inclusion of all limited English proficient students in its 

for developing them. assessment system. 

Interview: 

•Discuss the State’s definition of LEP. 

•Discuss the State’s criteria for student exit from the LEP 
accountability subgroup. Is this consistent with the 
Accountability Workbook? 

•What guidelines does the State have in place for including all 
students with limited English proficiency (LEP) in the tested 
grades in the assessment system 

•Does State have appropriate accommodations for LEP students? 

1.2 The SEA has implemented all required How are students with limited English proficiency included in 

components as identified in its the State’s definition of adequate yearly progress? 

accountability workbook. Documentation: 

•State provides data showing that all LEP students participate in 
statewide assessments: general assessments with or without 
accommodations, LEP alternate assessments, or a native 
language version of the general assessment based on grade 
level standards. 

•State provides to all LEAs guidance on the assessment of new 
immigrant LEP students that is consistent with ESEA. 

1.3 The SEA has published an annual report Does the SEA have procedures to monitor annual Local 
card as required and an Annual Report Educational Agency (LEA) Report Cards to ensure that 

to the Secretary. information in the report cards is based on statistically valid and 

reliable data? 

Documentation: 

• State conducts audit check of LEA data and has procedures to 
resolve errors. Most recent State Consolidated Report. Check 
for completeness. 

• Data in the State Consolidated Report matches Report Card 
data for the same year. 

1 .4 The SEA has ensured that LEAs have 
published annual report cards as 
required. 

1.5 How are students with limited English 
proficiency included in the State’s 
definition of adequate yearly progress? 

Documentation: 
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Table 2 

Summary of Title 1 Monitoring Indicators and Guiding Questions Pertaining to ELLs 

Component Description Guiding Questions Pertaining to ELLs 

•State provides data showing that all 
LEP students participate in statewide 
assessments: general assessments 
with or without accommodations, 

LEP alternate assessments, or a 
native language version of the 
general assessment based on grade 
level standards. 

•State provides to all LEAs guidance on 
the assessment of new immigrant 
LEP students that is consistent with 
ESEA. 

The SEA indicates how funds received 
under Grants for State Assessments and 
related activities (Section 6111) will be 
or have been used to meet the 2005-06 
and 2007-08 assessment requirements of 
ESEA. 


1.6 The SEA ensures that LEAs meet all • Does the State conduct an annual language survey that 

requirements for identifying and identifies the following? 

assessing the academic achievement of 1 . Languages other than English present in student 

limited English proficient students. population; and 

2. Languages for which yearly academic assessments are 
needed. 

Documentation: 

• Summary of home language survey results, OELA report, or 
relevant consolidated application data that shows the number 
of languages present and relative incidence. 


*For ease of comparison between the guidance issued for 2006-2007, 2007-2008, and 2008-2009, Table 2 uses the 6 indicators in the 2006- 
2007 Title I monitoring cycle and records the three additional indicators introduced in 2007-2008 as part of indicator 1.1. 

Source: U.S. Department of Education. (2009b). Title I guidance and monitoring reports. 


SEAs are monitored for Title I compliance every three years. The monitoring review includes a 
site visit to the SEA and selected LEAs. As shown in Table 3, during the most recently-completed 
three-year monitoring cycle, 17 SEAs were reviewed and received monitoring reports in 2006- 
2007; 16 in 2007-2008; and 19 SEAs in 2008-2009 (includes the District of Columbia (DC) and 
Puerto Rico (PR)). The new three -year monitoring cycle, begun in 2009-2010, is currently 
underway. 


Table 3 

Title I Monitoring Visits during the Three-Year Monitoring Cycle, 2006-2009 N=52 


2006-2007 (17 SEAs) 2007-2008 (16 SEAs) 2008-2009 (19 SEAs) 


State 

Onsite Visit 

State 

Onsite Visit 

State 

Onsite Visit 

New Jersey 

January 8, 2007 

Colorado 

October 22, 2007 

Kansas 

October 20, 2008 

Ohio 

January 8, 2007 

Florida 

November 5, 2007 

Oregon 

November 3, 2008 

Washington 

February 12, 2007 

Oklahoma 

December 3, 2007 

Mississippi 

November 17, 2008 

Massachusetts 

March 5, 2007 

New Mexico 

December 10, 2007 

DC 

December 8, 2008 

Missouri 

March 12, 2007 

Texas 

January 14, 2008 

South Carolina 

January 12, 2009 

Minnesota 

March 26, 2007 

Maryland 

February 1 1 , 2008 

Georgia 

January 26, 2009 

Nevada 

April 16, 2007 

Arizona 

February 25, 2008 

Louisiana 

February 9, 2009 

Rhode Island 

April 23, 2007 

Pennsylvania 

March 3, 2008 

Tennessee 

February 23, 2009 

Nebraska 

May 7, 2007 

Idaho 

May 5, 2008 

Puerto Rico 

March 2, 2009 

Michigan 

May 14, 2007 

Illinois 

April 14, 2008 

Arkansas 

March 9, 2009 

Maine 

June 4, 2007 

Wisconsin 

April 21, 2008 

New York 

March 23, 2009 
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Table 3 

Title 1 Monitoring Visits during the Three-Year Monitoring Cycle, 2006-2009 N=52 


2006-2007 (17 SEAs) 2007-2008 (16 SEAs) 2008-2009 (19 SEAs) 


State 

Onsite Visit 

State 

Onsite Visit 

State 

Onsite Visit 

Montana 

South Dakota 

California 

Connecticut 

Iowa 

Indiana 

June 4, 2007 
June 11,2007 
August 13, 2007 
September 10, 2007 
September 10, 2007 
September 24, 2007 

North Dakota 
North Carolina 
Virginia 
Alabama 
Delaware 

May 5, 2008 
May 19, 2008 
June 9, 2008 
September 8, 2008 
September 15, 2008 

Hawaii 
Utah 
Alaska 
Kentucky 
West Virginia 
Wyoming 
New Hampshire 
Vermont 

April 20, 2009 
April 27, 2009 
May 4, 2009 
May 18, 2009 
June 1, 2009 
September 14, 2009 
September 21, 2009 
September 21, 2009 


Data Collection and Analysis 

The research team reviewed two sets of documents taken from the U.S. Department of Education 
Weh site (www.ed.gov) . These included 273 peer review decision letters issued to 52 SEAs from 
June 17, 2005 to January 15, 2009. Multiple letters were issued to most SEAs. Title I monitoring 
reports issued to the 52 SEAs (DC and PR included) during the most recently-completed three- 
year cycle, occurring in 2006-2007, 2007-2008, and 2008-2009, were reviewed. 

Qualitative analysis was used to examine the two sets of reviews. In the first stage, data 
pertaining to ELLs were sorted and counted according to the peer review components or Title I 
indicators prescrihed for each type of review. Next, inductive analysis was used to generate the 
inclusion and accommodation issues related to ELLs in state assessments. The accommodation 
issues identified in peer review feedback and Title I monitoring reports were similar to those 
identified in the 2007 Christensen and Thurlow study of peer review feedback concerning 
students with disabilities. SEAs receiving feedback on an issue were noted. Errors in reviewers’ 
categorizations were noted and corrected. Examples of errors included accommodations feedback 
misclassified under the wrong component, and lack of clarity about whether the population 
addressed in the feedback referred to ELLs or students with disabilities. 

Limitations of this Study 

The data examined for this study are influenced by the limitations of the individual procedures 
established for each review, the quality of the data provided by SEAs for reviewers, and the 
quality of the feedback provided by reviewers. 

The uneven and inconsistent recommendations and counsel given to SEAs in the two types of 
review may be, in part, due to the different purposes and procedures for providing feedback, the 
frequency of reviews, and the expertise of those who conduct the reviews. Eor example, while 
peer reviews (and the resulting decision letters) provide cumulative feedback on an SEA’s 
standards and assessment system. Title I monitoring reports provide feedback on the 
implementation of procedures and practices related to Title I implementation at one point in time 
based on priorities set in a given year by ED, which may or may not address the inclusion and 
accommodation of ELLs. Title I monitoring reports require that data be prepared in advance of 
the visit and during the site visit to the SEA and LEAs. Einally, Title I monitoring visits are 
completed by ED staff, not outside experts. 

The quality and variety of data provided by the SEAs for the reviews also has an impact on the 
quality of feedback. Because SEAs vary in the number of ELLs served, the comprehensiveness of 
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programs offered, and the expertise of the individuals preparing the information for reviewers, the 
quantity and quality of information offered to peers and Title I monitors vary. While evidence 
provided hy some SEAs may directly address the assessment of ELLs, other SEAs may not 
directly address ELLs and/or may bundle the information with data for students with disabilities. 
This variability may make it difficult for reviewers to assess or provide targeted feedback about 
the inclusion and accommodation of ELLs. 

Einally, the quality of the feedback provided by reviewers is a limitation. While one of the three 
peer review committee members is selected on the basis of expertise in issues affecting either 
students with disabilities or ELLs, there is no guarantee that a member of the committee, and thus 
the team, will have expertise to address both student populations. Like peer reviewers, the 
individual Title I monitoring team members and the team as a whole may or may not have 
expertise in the assessment of ELLs. 

Findings 

Leedback regarding ELL inclusion and accommodation from peer review decision letters and 
Title I monitoring reports varied across states. The amount and types of feedback differed for the 
two data sources, reflecting the differing purposes of the two types of review. In the area of 
assessment, peer review focused primarily on assessment design, while Title I monitoring 
emphasized the implementation of assessments. 

As shown in Table 4, roughly half of SEAs received feedback concerning ELL inclusion 
practices in Title I monitoring reports and peer review decision letters. While SEAs were more 
likely to receive feedback on ELL accommodation practices in peer review decision letters than 
in Title I monitoring reports, almost three-quarters of SEAs received requests for evidence or 
recommendations about the accommodation of ELLs in peer review decision letters, compared 
with less than a quarter of SEAs receiving accommodations feedback in Title I monitoring 
reports. 


Table 4 


Number of SEAs Receiving Feedback on the Inclusion and Accommodation of ELLs in Peer Review Decision Letters 
and Title I Monitoring Reports 


Source of Feedback 

ELL Inclusion 

ELL Accommodation 

Peer Review Decision Letters 

26 of 52 SEAs (50%) 

38 of 52 SEAs (73%) 

Title I Monitoring Reports 

30 of 52 SEAs (57%) 

10 of 52 SEAs (19%) 


Issues Pertaining to the Inciusion of ELLs 

In each review process, approximately half of the states received feedback on inclusion issues. In 
peer reviews, 26 of 52 SEAs (AK, CO, CT, DC, HI, lA, ID, IL, KS, KY, MD, MN, MS, MT, ND, 
NE, NH, NM, NY, PR, SD, TX, VA, VT, WA, WY) received one or more requests for evidence 
or recommendations concerning ELL inclusion. In Title I monitoring reports, 30 of the 52 SEAs 
monitored (AK, AR, CA, CO, DC, EL, GA, HI, lA, ID, IL, KS, KY, LA, MI, MO, MS, MT, ND, 
NE, NJ, NY, OH, OK, PA, PR, RI, VT, WV, WY) received one or more recommendations or 
findings concerning ELL inclusion. Sixteen SEAs (AK, CO, DC, HI, lA, ID, IL, KS, KY, MS, 
MT, NE, NY, PR, VT, WY) received ELL inclusion feedback from both sources. 
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In the analysis, four inclusion issues emerged. Two were common across both the peer review 
decision letters and the Title I monitoring reports and two issues were found only in the Title I 
monitoring reports: 

• How ELLs were defined in state policy for inclusion in assessments, 

• Methods used to count ELLs for AYP, 

• SEA monitoring of LEA inclusion practices, and 

• SEA reporting in state or district report cards of the participation of ELLs in state 
assessments. 

The ways in which state policies defined ELLs for purposes of inclusion in assessments was 
addressed in peer review decision letters to 12 SEAs (CT, KS, MN, ND, NH, PR, RI, SD, TX, 
VA, VT, WA) and in Title I monitoring reports to six SEAs (KY, lA, MS, OK, PR, VT). Peer 
review requests for evidence in this category focused on policy language or guidance to clarify 
the extent to which ELLs were being included in state assessments, such as for recently-arrived 
ELLs and migrant students. In Title I monitoring reports, the six SEAs received one or more 
findings or recommendations concerning the need to improve the SEA’s operational definition of 
an ELL in state assessment policy. 

The methods used to calculate ELL participation for AYP purposes were found by reviewers to 
be problematic for 27 SEAs. Twenty-three SEAs received general feedback on this issue in their 
peer review letters (AK, CO, CT, DC, HI, lA, ID, IL, IN, KS, KY, MD, MN, MO, MS, MT, NE, 
NM, NY, SD, TX, VA, WY) and eight SEAs received this feedback in Title I monitoring reports 
(CA, lA, ID, MS, NE, OR, PR, RI). Pour states (lA, ID, MS, NE) received feedback on the issue 
in both peer review and Title I monitoring reports. Peer review and Title I monitoring report 
feedback focused on the numbers of students enrolled, the numbers of students assessed, and the 
numbers of students excluded by grade and subgroup within each content area. Por example, peer 
review feedback for Alaska requested “evidence that all students in the grades tested are included 
in the science assessments.” This general statement was a common way this request was framed 
in state peer review letters. In its Title I monitoring report, California received a recommendation 
to change the way it counts ELLs when calculating AYP to conform to the ED regulation (U.S. 
Department of Education, 2007) governing how ELLs and former ELLs were included in the LEP 
subgroup. Iowa received several findings in its September 2007 Title I monitoring report which 
required further action to improve AYP calculations, specifically in relation to categorizing ELLs 
as “full, transition and exited.” Rhode Island received a citation after its April 23-27, 2007 Title I 
monitoring visit, with further action required to provide LEAs with a clear definition of the data 
elements required for calculating AYP. 

SEA monitoring of LEA inclusion practices was found to be problematic by reviewers only in 
Title I monitoring reviews. Specifically, the reliability of LEA implementation of the SEA 
definition of an ELL for assessment purposes was identified by reviewers in 1 1 SEAs (AK, CO, 
PL, lA, IL, MS, MT, ND, NJ, OK, and RI). The analysis of reviewer comments indicated that (1) 
LEAs in some states were not aware of criteria for defining students as ELL/LEP; (2) some SEAs 
permitted LEAs to determine their own criteria for excluding ELL/LEP students from the state 
assessment; and (3) some SEAs permitted LEAs to determine their own criteria for exiting 
students from LEP status. 

Pinally, also addressed only in Title I monitoring reviews was SEA reporting of ELL 
achievement in state and/or district report cards. Reviews indicated a need to increase the 
accuracy of reports of ELL achievement on specific assessments in state and district report cards. 
In all, 21 SEAs received one or more findings or recommendations to improve reports of ELL 
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achievement in state and/or LEA report cards, with more emphasis placed on this issue in the 
later years of the three-year monitoring cycle. While only two SEAs received this feedback 
during the first year of the three-year monitoring cycle (MI, OH); seven SEAs received feedback 
during the second year, 2007-2008 (CO, EL, ID, MO, MT, OK, PA); and 12 in the third year, 
2008-2009 (AR, DC, GA, HI, KS, KY, LA, MS, NY, PR, WV, WY). 

Issues Pertaining to the Accommodation of ELLs 

Eorty SEAs received feedback about the accommodation of ELLs. SEAs received more feedback 
on accommodation issues in peer review decision letters than in Title I monitoring reports. As 
shown previously in Table 4, 38 SEAs received ELL-related peer review requests for evidence or 
recommendations related to technical quality of the accommodations (AK, AR, CA, CO, CT, DC, 
GA, HI, lA, ID, IL, KS, KY, LA, MD, ME, MI, MN, MO, MS, MT, NC, ND, NH, NJ, NM, NV, 
NY, OR, PA, PR, RI, SD, TX, VT, WA, WI, WY). Title I monitoring reports, in contrast, 
mentioned these types of issues far less frequently. Overall, 10 SEAs (AK, EL, GA, lA, IL, KY, 
MT, NM, OK, and RI) received feedback on accommodations issues from Title I monitoring 
reports. Alaska, Georgia, Iowa, Illinois, Kentucky, Montana, New Mexico, and Rhode Island 
received feedback about accommodations in both peer review decision letters and Title I 
monitoring reports. 

Eeedback in this category addressed four issues: 

• Validity and meaningfulness of accommodated ELL scores, 

• Practices for monitoring the implementation of ELL accommodations, 

• ELL accommodations allowed in state policy, and 

• Alignment of assessment accommodations and instructional practices. 

The most common of the four issues, validity and meaningfulness of accommodated scores, is 
described first. Thirty-two SEAs (AK, AR, CA, CO, CT, DC, HI, ID, IL, KS, LA, MD, ME, MI, 
MN, MO, MS, MT, ND, NH, NJ, NM, NV, NY, PA, PR, SC, SD, VT, WA, WI, WY) received 
feedback from peer reviewers. Most peer review requests were for results of studies and/or 
evidence that research was being conducted. Examples included requests for “evidence that the 
State has evaluated the use of the accommodations permitted,” that the use of accommodations 
“results in valid scores and that those scores can be aggregated with scores from non- 
accommodated assessments,” and that “the use of accommodations yields meaningful scores.” 
The Mississippi decision letter was framed in relation to the Standards for Educational and 
Psychological Testing (AERA/APA/NCME, 1999), while the District of Columbia decision letter 
outlined a three-phase process to examine prior use of accommodations, to study 
accommodations in relation to the design of the assessment, and to develop training guidelines 
and policies for implementing accommodations. Illinois was the only state to receive feedback in 
its Title I monitoring report on the validity of accommodated scores. The SEA was asked to 
explain how the read-aloud script for its high school assessment provided meaningful scores and 
valid inferences about student knowledge and skills and would not invalidate the reading 
construct. 

In all, twenty-four SEAs received feedback related to the need to monitor the implementation of 
ELL accommodations. Twenty-one of these SEAs (DC, GA, lA, ID, IL, KS, KY, MD, MN, MO, 
MS, NC, ND, NH, NJ, PA, RI, SD, TX, VT, WI) received peer review requests for evidence or 
recommendations regarding their practices for monitoring the availability and use of 
accommodations by ELLs. In their Title I Monitoring reports, four SEAs received feedback 
regarding the procedures used for monitoring accommodations availability and use during testing 
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(FL, lA, MT, OK). For example, in peer review feedback, Pennsylvania was asked for additional 
evidence on “final (not draft) assessment system and testing activities' monitoring plan with 
additional elements focused on auditing limited English proficient (LEP) accommodations” and 
Maryland received a request for “documentation that the monitoring of accommodations is 
occurring.” In its October 2008 decision letter, the District of Columbia received a plan detailing 
monitoring data to be collected. Reviewers recommended that DC show the number and 
percentage of students who took an assessment with particular accommodations and compare 
accommodations available to ELLs with accommodations actually used. After ED’s November 5- 
9, 2007 Title I monitoring visit to the Elorida Department of Education (EDE) there was a 
recommendation to clarify how accommodations are determined - whether by a decision-making 
team or through a general requirement that all ELLs receive accommodations. In its Title I 
monitoring report, Montana was asked to monitor the specific accommodations used by LEAs 
since a visit to one LEA showed only one accommodation (extended time) being offered to ELLs. 

Another issue identified pertained to the specific accommodations allowed for ELLs in SEA 
assessment policies. Eight SEAs (DC, Eli, MT, ND, NM, PR, WI, and WY) received feedback 
from peer reviewers and five states (AK, GA, KY, NM, and RI) received feedback in Title I 
monitoring reports. Eight SEAs received requests from peer reviewers for additional evidence or 
recommendations regarding the consistency and accuracy of specific accommodations, e.g., 
scripted read aloud accommodation in two states (DC, ND), assessment translations (WI and 
WY), and English versions and simplified Spanish versions of the mathematics and science 
assessments (PR). Additionally, during peer review, three SEAs received requests for additional 
evidence about state policy or procedures for specific accommodations (HI, MT, and NM). 
Hawaii was asked for evidence of “procedures for the standardization of the accommodation that 
allows for the explanation of directions using simplified vocabulary.” Montana was asked to 
“provide a clear distinction among which accommodations are allowable for students with lEPs, 
students with Section 504 plans, and ELLs.” Like its peer review feedback. New Mexico Public 
Education Department’s December 2007 Title I monitoring report stated that it must “submit to 
ED criteria for oral translations to ensure the validity and reliability of these translations and a 
training schedule for the LEA test coordinators to ensure the implementation of these criteria or 
revised guidance that eliminates this accommodation.” In Rhode Island’s Title I monitoring 
report (April 2007) the addition of side-by-side English/Spanish versions of the math NECAP 
and/or audio-recorded test directions” was recommended as long as the SEA could document that 
the accommodations would not invalidate the test construct. In the final year of the three-year 
Title I monitoring cycle, three states (AK, GA, and KY) received feedback on the general quality 
of information available in state policies to LEAs concerning the selection of accommodations. 

A final issue, the alignment of assessment accommodations and instructional practices, emerged 
only in the peer review decision letters of three SEAs (HI, MT, NJ). Reviewers wanted to know if 
Hawaii’ s read-aloud accommodation was allowed only for students receiving the accommodation 
in instruction (October 30, 2007 letter). As in Montana’s June 22, 2006 decision letter. New 
Jersey’s September 2008 decision letter requested evidence “that appropriate accommodations are 
available to students with disabilities and limited English proficient (LEP) students and that these 
accommodations are used in a manner that is consistent with instructional approaches for each 
student determined by a student's lEP or 504 plan.” 

In summary, two significant findings emerged from this study. The primary finding is that the 
majority of SEAs have weaknesses in their policies and practices for including and 
accommodating ELLs in state assessment systems. An additional important finding is that the 
feedback itself is inconsistent both within and across the peer review and Title I monitoring 
processes. 
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Discussion 

To restate, the goal of this study was to examine the types of feedback provided to SEAs in peer 
review letters and Title I monitoring reports about their policies and practices for including and 
accommodating ELLs in state assessments. Overall, the analysis revealed that feedback to SEAs 
on the inclusion of ELLs in state assessment was slightly less common than feedback on 
accommodation practices. 

With regard to inclusion, the issues identified in ED feedback addressed: 

• How ELLs were defined in state policy for inclusion in assessments, 

• Methods used to count ELLs for AYP, 

• SEA monitoring of LEA inclusion practices, and 

• SEA reporting in state or district report cards of the participation of ELLs in state 
assessments. 

In each of these areas, feedback by reviewers to states was scattered and, when given, 
inconsistent. Only 18 states received feedback related to their definition of ELL. Because states 
do not have common methods for defining this student population, students classified ELL in one 
state may not be classified ELL in a neighboring state. This is complicated further by the fact that 
in some states the definition of ELL can vary by district (Linquanti & George, 2007). This 
definitional inconsistency within and across states is problematic in general. Because SEAs 
provide different types of evidence to reviewers and because of the different purposes of the two 
review processes, their response to the protocols for the two reviews may miss or only obliquely 
address key points related to the inclusion and accommodation of ELLs. 

With regard to how ELLs are counted for accountability purposes, 27 SEAs received feedback. 
The number of states that received feedback suggests that reviewers were attuned to the issue that 
unpredictability in classification can negate the accuracy of the AYP count for ELLs (Abedi & 
Dietel, 2004). In this case, the feedback signals that states may not be providing clear information 
about the overall number of ELLs in the state and the actual number of students tested in state 
assessments. 

SEA monitoring of LEA practices was found to be problematic in 1 1 state Title I monitoring 
reviews. Although the majority of states acknowledge not having or having only a minimal 
monitoring system in place, reviewers provided feedback to states about monitoring 
systems/practices on a limited basis. This is likely due to the type of evidence provided by the 
states, other issues raised in the review, and the expertise of the reviewers themselves. 

Linally, in the monitoring review, it was found that nearly half of SEAs (21) were cited for not 
providing specific information about the number of ELLs and their participation in state 
assessments in state and/or district report cards. This lack of information may be due to the fact 
that states are not uniformly collecting data from LEAs. Lacking specific and accurate 
information about LEA practices, the SEA lacks the data needed to report on ELL participation in 
state assessments. The inconsistent reporting of evidence makes it difficult for peers and Title I 
monitors to provide meaningful or critical feedback on how a state is progressing in this area. 

Overall, inclusion issues addressed in peer review letters and Title I monitoring reports suggest 
the need for SEAs to reassess their policies and to find methods to uniformly implement them 
across a state. The fact that feedback about the inclusion of ELLs was given only to some states 
also suggests that perhaps peers and Title I monitors should be required to comment on the topic 
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for all states reviewed. Currently feedback is primarily provided on deficient practices and not on 
practices in which SEAs are improving or doing well. 

SEAs received the most feedback regarding the accommodation of ELLs. The issues addressed 
included: 

• Validity and meaningfulness of accommodated ELL scores, 

• Practices for monitoring the implementation of ELL accommodations, 

• ELL accommodations allowed in state policy, and 

• Alignment of assessment accommodations and instructional practices. 

The most feedback given to SEAs concerned the need to conduct research on accommodations 
(32 SEAs) to ensure that accommodated test scores can be shown to be valid and, therefore, 
pooled with unaccommodated test scores. Considering that research on accommodations for 
ELLs is thin, the recommendation is reasonable. Only 14 experimental studies, to date, 
specifically address the accommodation of ELLs, so much more focused research is needed 
(Erancis, Rivera, Lesaux, Kieffer, & and Rivera, 2006; Kieffer, Lesaux, Rivera, & Erancis, 2009; 
Pennock-Roman & Rivera, 2007). 

Monitoring the selection and implementation of accommodations in state assessments was 
another focus for reviewers. Almost half of the SEAs received feedback that they lacked an 
adequate system for monitoring the selection and implementation of accommodations for ELLs 
(24 SEAs). The responses to the GW-CEEE SEA monitoring survey also indicated that SEAs 
recognize that current systems for monitoring ELL assessment are non-existent or inadequate. 
Most SEAs recognized that they do not have consistent or cohesive policies or systems in place 
for collecting data to track the implementation of accommodated assessments for ELLs. These 
data are needed to demonstrate the validity of accommodated scores and to contribute to 
improved implementation of appropriately accommodated assessments for ELLs. Because of the 
lack of data, SEAs, researchers, and the federal government lack needed information to support 
improvements in SEA practices. Continued focus on this topic is critical. 

Three SEAs received feedback about the need to align assessment accommodations with 
accommodations received by the student in instruction. This feedback is problematic from two 
perspectives. One, it suggests, and that SEAs do not have accommodation policies that clearly 
distinguish accommodations intended for ELLs from accommodations intended for students with 
disabilities. Second, current peer review guidance calls for the alignment of instructional and 
assessment accommodations. Yet the term instructional accommodations is a designation in 
IDEA specifically applied to students with disabilities as part of a student’s individual education 
program (lEP). No similar federal requirement applies to ELLs, thus, when ELLs are tested with 
other students, there is no mechanism to know which accommodations are or have been used in 
instruction. Also, ESL and other teachers of ELLs do not commonly refer to instructional 
accommodations; rather, they use terms such as scaffolding and differentiated support. While 
many supports provided to ELLs as part of classroom instruction and assessment might also be 
appropriate accommodations on state assessment, states should not be held accountable for 
documenting the alignment of instructional strategies and assessment accommodations for ELLs 
unless the requirement is part of federal and/or local law. (Only a handful of states such as 
Kentucky and Idaho have included this requirement in their state regulations.) The issue elicits 
two reactions: one for states to more clearly articulate accommodation policy for ELLs and 
another for reviewers to become more familiar with linguistic accommodations intended for ELLs 
that are ELL-responsive (Rivera, Collum, Shafer Willner, & Sia Jr., 2006; Rivera, Acosta, & 
Shafer Willner, 2008). 
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The wide range of ELL inclusion and accommodation feedback and the lack of similarity and 
detail in feedback given to SEAs points to a lack of consistency and coherence in the evidence 
provided to ED by SEAs as part of peer review and Title I monitoring. In some cases, it also 
suggests that review teams need additional background to distinguish accommodations intended 
for ELLs versus students with disabilities. 

The lack of consistency and coherence in the data provided by SEAs for the two reviews suggests 
a need for a concerted effort to clarify for states what is considered acceptable evidence. Eindings 
from the study would suggest that SEAs should be required at a minimum: 

• to provide demographic data and state policy for including ELLs in assessment, 
including the data on the number of ELLs included in or excluded from state 
assessments; 

• to provide a rationale for why students are excluded from an assessment; 

• to explain whether LEAs are required to use the state policy when making decisions 
about including ELLs in state assessment; if LEAs have liberty to use their own 
criteria, SEAs should be required to provide criteria used by one or more large LEAs; 

• to provide state policy for accommodating ELLs; 

• to demonstrate the state has conducted, or is planning to conduct, a study on the effect 
of one or more of the accommodations being used; and 

• to provide a description of the assessment monitoring system for all students and to 
highlight practices for ELLs. 

The findings also have important implications for the two review processes. The inconsistencies 
within and across standards and assessment peer review decision letters and Title I monitoring 
reports point to the need for greater alignment between peer review and Title I monitoring. The 
fact that several SEAs received feedback on their ELL policies and practices in peer review 
regarding their science assessment but not on reading/language arts and mathematics assessments 
suggests that there may now be more attention to the assessment and accommodation of ELLs. 

Einally, the study findings indicate a need to select reviewers familiar with the assessment of 
ELLs and to train review teams so all members have a common understanding of issues related to 
the inclusion and accommodation of ELLs. This supports the need for every team to include an 
ELL assessment expert even if it means increasing the size of the team. 

Recommendations 

Based on the findings and discussion, recommendations for SEAs providing data for review and 
for ED when conducting peer review and Title I monitoring follow. 

Recommendations for SEAs providing data for review 

• Provide demographic data; document the number of ELLs and the number of ELLs 
included in each state assessment. 

• Provide state policy and procedures for accommodating ELLs; clarify whether LEAs 
have local control or are subject to state policy. 

• Document SEA written guidance and procedures for LEAs to collect data and report 
inclusion and accommodation data to the SEA. 
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• Document any program of research in place carried out by the SEA independently or in 
partnership with other states on the effectiveness of specific accommodations allowed 
to ELLs. 

Recommendations for ED when conducting peer review and Titie I monitoring 

• Improve alignment of the peer review and Title I monitoring rubrics making clear what 
ELL issues are to be addressed by SEAs and reviewers. 

• Make clear expectations for each review process with regard to addressing the 
inclusion and accommodation of ELLs. 

• Align peer review and Title I monitoring guidance so that the accommodation of ELLs 
is addressed as a technical quality issue and not as an issue uniquely pertaining to 
ELLs. 

• Refine the language in both peer review guidance and in the Title I monitoring rubric 
so it clearly distinguishes the needs of ELLs from students with disabilities. 

• Provide Title I monitors with access to peer review feedback, so the two reviews are 
complementary and build on each other. 

• Include experts in the assessment of ELLs on all peer review committees and Title I 
monitoring teams. 

• Require reviewers to record whether an SEA has addressed each issue on a protocol; 
both acceptable and unacceptable practices should be documented. 

Conclusion 

In investigating the extent to which standards and assessment peer review and Title I Monitoring 
review processes provide consistent feedback to SEAs about their policies and practices for 
including and accommodating ELLs in state assessments, areas of overlap and areas where the 
reviews could be aligned and coordinated were clearly identified. Because SEAs’ standards and 
assessment systems are at different stages of development and different levels of complexity, this 
analysis points to a number of suggestions for improving the coherence, quality, and effectiveness 
of both SEA and ED practice. The implementation of the recommendations, based on the findings 
of this study, would likely lead to a greater understanding and, therefore, improved compliance of 
SEAs, more coherent SEA plans, and ultimately more meaningful inclusion and accommodation 
of ELLs in state assessment. 
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